-
Notifications
You must be signed in to change notification settings - Fork 6k
ollama: Fix tool calling #42275
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ollama: Fix tool calling #42275
Conversation
|
@tidely great work! your PR is more simplistic (not introducting logging) and as you mention not backwards compatible as the id is not optional. i think i'll add a test like you did so it covers both types of possible inputs. May i ask how - if you did not have logging - you discovered that? I must admit that i had a really hard time to figure out what was going on. |
I think the error already gets logged in high level code after the error bubbles up. Not sure how useful the errors are anyway, I think it might be worth exploring using the serde_path_to_error crate already used in other places in the Zed codebase. We should definitely merge our efforts on this though.
Sounds good!
I compared the request schema used in tests to the one returned by curling the newest ollama version, and then I did some digging in the Ollama repo |
|
@tidely note sure how to effectively continue. i think the issue you referenced is good. i focused on adding logging (debug&trace) for the ollama module which is great when you want to see how the agent interacts but effectively this is not really related to the issue. |
|
As per your recommendation @js-0s, I've made the PR backwards compatible with older Ollama versions and added a test for this behavior. Initially I thought we should just embrace the new response format and ditch older Ollama installations, as we haven't really concerned ourselves with Ollama backwards compatibility before either. It would've allowed us to remove some custom logic we use for our own internal tool calling identifiers and brought Ollama more in unison with other providers. But I figured some well documented TODO's and comments will allows us to strip those bits away later as well. Thanks for your input here and your efforts with your own PR as well! |
bennetbo
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice work, thank you!
Closes #42303
Ollama added tool call identifiers (ollama/ollama#12956) in its latest version v0.12.10. This broke our json schema and made all tool calls fail.
This PR fixes the schema and uses the Ollama provided tool call identifier when available. We remain backwards compatible and still use our own identifier with older versions of Ollama. I added a
TODOto remove theOptionaround the new field when most users have updated their installations to v0.12.10 or above.Note to reviewer: The fix to this issue should likely get cherry-picked into the next release, since Ollama becomes unusable as an agent without it.
Release Notes: